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EXECUTIVE SUMMARY 



Policymakers, higher education associations, blue-ribbon commissions, and researchers 
are calling for a greater focus on institutional accountability. Thus, the American 
Association of Community Colleges and the Association of Community College 
Trustees, in partnership with the College Board, have launched an effort to develop a 
Voluntary Framework of Accountability (VFA) for Community Colleges to ensure that 
the effectiveness and contributions of these colleges are properly measured. 

To facilitate the development of the VFA, the College Board commissioned the 
Community College Research Center (CCRC) to undertake a study that would identify 
the performance measures that states are already using for their community colleges, 
explore how well those measures articulate with the data demanded by IPEDS and the 
regional accrediting associations, and shed light on the experiences of state higher 
education officials and local community college leaders with the collection and use of 
state performance data. CCRC researchers therefore conducted a survey of state higher 
education officials and local community college leaders in 10 states and also reviewed 
publications by state agencies and national higher education policy organizations. 

Study Findings 

Commonly used indicators 

The 10 states in CCRC’s study together collect data on as many as 140 specific indicators 
of community college performance. The most common indicators are the following: 

(1) Input indicators , such as total enrollments; populations served (for example, the 
proportion of area adults enrolled); and enrollments disaggregated by student 
demographics. 

(2) Process indicators, such as tuition and fees; expenditures; faculty characteristics; 
and the extent of vocational, remedial, dual-enrollment, and online programming. 

(3) Output indicators, such as rates of remedial and developmental education success; 
graduation from the community college; transfer to a four-year college or 
university; and employment after leaving the community college. 

There is a relative dearth of indicators broken down by student background, particularly 
income, gender, age, and immigration status. This inattention to demographic 
breakdowns is particularly pronounced for output/outcome indicators. It is striking as 
well that overall student learning does not get more attention in the state performance 
accountability systems in the 10 states surveyed. 
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Compatibility with IPEDS and regional accreditation demands 

Our state respondents perceived a relatively low fit between the data demands of state 
performance accountability systems and IPEDS. However, they perceived a stronger fit 
with the data demands of regional accreditors. 

Data reporting 

The state performance data are publicly reported in all 10 of the states, with publication 
on the internet as the preferred dissemination method. All 10 states break down the 
performance data by institution, but they differ in how they treat the resulting differences 
among colleges. Some states group the colleges by peer groups based on shared college 
characteristics and, sometimes, by community characteristics. 

Data collection issues 

Only a few of the state officials interviewed reported significant difficulties in securing 
performance data from the colleges. However, local community college officials have a 
more jaundiced assessment of the data collection process, focused on the difficulties of 
acquiring certain data that the state demands. 

Use of community college performance data at the state and local levels 

For the most part, state higher education officials stated that the performance data did not 
significantly affect state decisions. In fact, local community college officials noted that 
state use of performance data was characterized largely by a lack of substantial scrutiny 
of performance data from the colleges. Most of the local community college officials 
stated that they do use the data they report to the state in making local decisions. 
However, some colleges reported that they do not rely very much on the state 
performance data but instead rely much more on their own internally generated 
performance data. Our local respondents noted a number of impediments to their 
effective use of state performance data. 

Recommendations 

The findings from CCRC’s study have definite implications for efforts to construct a 
Voluntary Framework for Accountability for Community Colleges. 

Performance indicators to consider 

Any indicator system should have a balanced combination of input, process, and 
output/outcome indicators for individual colleges and systemic indicators for the entire 
higher education system. The input indicators should provide incentives for maintaining a 
focus on enrolling less advantaged students. The process indicators should reward efforts 
to be accessible and to provide curricular and other programming that prepares students 
broadly and deeply. And the output/outcome indicators should stimulate colleges to focus 
on socially important outcomes for all students. Systemic indicators, meanwhile, would 
capture outcomes that are the product of the entire higher education system. 
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States should include among their input indicators measures for access by students of 
different backgrounds, including race/ethnicity, income, gender, age, and immigration 
status. The presence of such indicators can help counterbalance the incentive that 
graduation and job placement indicators create for colleges to focus on enrolling more 
advantaged students since they are more likely to graduate. 

Four kinds of process indicators deserve attention: 

(1) How accessible community colleges make themselves to less advantaged students, 
through, for example, low tuition and fees. 

(2) Whether community colleges offer a broad range of programs addressed to a 
variety of student needs and interests, whether through high school completion 
(GED, etc.) programs, remedial education, preparation for transfer, or workforce 
development. 

(3) Community colleges’ provision of quality learning, indexed by such factors as 
degree of student academic engagement. 

(4) Availability of resources for community colleges, such as sufficient funding per 
full-time equivalent (FTE). 

The output or outcome indicators clearly should address eventual success as measured by 
graduation, transfer, and job placement. However, it also important to keep in mind that: 

(1) Summary indicators of community colleges outcomes — such as total number of 
students completing — should include not just graduation but also transfer. 

(2) Colleges need to be rewarded for how well they are doing on intermediate 
outcomes such as completion of GED and remedial education programs and 
passage of important gatekeeper courses such as college-level English and 
mathematics. 

(3) There should be measures for how well students are completing, through 
indicators for post-transfer performance, passage of licensure exams, employer 
satisfaction with graduates, and long term job stability. 

(4) Beyond specific measures of student learning, consideration should also be given 
to general student learning, with faculty playing an important role in the design of 
indicators and measures. 

(5) Just as the input indicators need to be broken down by student background, so do 
the output or outcome indicators. 



Finally, attention should be given to securing good indicators of systemic performance: 
that is, how well the entire higher educational system is creating cultivated citizens and 
technically prepared workers. 




Ways to better measure indicators 



In order to yield data that validly measure how community colleges are performing, it is 
important that the indicators above be properly measured and that the resulting 
information be put in the right context. 

Proper measurement. It is important that the data collected pertain to the relevant 
students and fully cover their performance. Data should be gathered on all students and 
not just on those who are attending full time or begin their enrollment in the summer or 
fall. Moreover, it is important that the data on graduation rates pertain to students who are 
first-time students in community college and are indeed degree seeking. Also, every 
effort should be made to count all students who transfer to a four-year college: those 
transferring with or without a degree, to private as well as public colleges, and to out-of- 
state as well as in-state institutions. Finally, the time frame for tracking outcomes for 
students should be extended to at least six years after entry. 

Contextualization of data. Community colleges vary greatly in student composition and 
the nature of their catchment areas. Therefore, performance indicators for community 
colleges need to be contextualized or benchmarked, either by comparing colleges to 
relevant peer groups or by statistically adjusting performance data for the entry 
characteristics of students. Some contextualization can be achieved by calculating 
separate completion rates for remedial and non-remedial students and comparing colleges 
within each category. In addition, performance measures must take into account local 
labor market differences, acknowledging the difficulties of some colleges — particularly 
in rural areas — in meeting demands for job placement in well-paying jobs. Finally, 
small colleges need to be cushioned against the distortive effects of small sample sizes, 
where changes in only a few students can have a dramatic effect on average performance. 
One way to counteract the effects of sampling fluctuation would be to utilize rolling 
averages (as is done in Tennessee and North Carolina). 

Ways to make collection of performance data easier for community colleges 

Community colleges identified the following initiatives by states would help address the 
problems in collecting and reporting performance data: 

(1) Data requirements should be tailored as much as possible to data already collected 
for state student unit record systems. 

(2) Community colleges would benefit from having the state itself take over more 
collection of data from graduate and employer surveys and from “third parties” 
such as the state university system and professional associations. 

(3) For data that they must still collect and analyze for themselves, community 
colleges would benefit from state-provided technical assistance and training and 
help in acquiring better information technology capacity. 



IV 




Ways to encourage better use of performance data 

CCRC’s interviews point to several ways that analysis of college performance data could 
be improved for the benefit of both the colleges and their stakeholders: 

(1) Assistance for colleges in improving their information technology and 
institutional research capacity to analyze data. 

(2) Incentives for college constituents — particularly faculty and middle managers — 
to pay greater attention to performance data. 

(3) An increase in the number and skills of state staff members and officials who can 
analyze and comment on the performance data that colleges submit. 

(4) Incentives to legislators and executive branch officials to make greater use of 
performance data when making policy decisions. 
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INTRODUCTION 



Over the past three decades policymakers have become very concerned about finding 
ways to secure better performance from higher education institutions, whether in the form 
of greater access and success for less advantaged students, lower operating costs, or 
improved responsiveness to the needs of state and local economies. As a result, great 
effort has gone into designing incentives for improved college performance. 

One of the key incentives that policy analysts have argued for and state governments 
have tried is a state performance accountability system, whether in the form of requiring 
reports on performance outcomes (performance reporting) or tying state funding to an 
institution’s performance on specific indicators such as rates of retention, graduation, and 
job placement (performance funding and performance budgeting) (Alexander, 2000; 
Burke, 2004; Burke & Associates, 2002; Dougherty & Hong, 2006; Ewell & Jones, 2006; 
Wellman, 2002; Zumeta, 2001). Today, virtually all states have some form of 
performance reporting and some 15 states have performance funding (Burke & 
Minassians, 2003; Dougherty & Reid, 2007). 1 

We are now entering a period of renewed interest in college performance. A variety of 
prominent higher education commissions and researchers are calling for a greater focus 
on performance accountability, though often in forms different from past practice 
(Blanco, Jones, Longanecker, & Michelau, 2007; Callan, Ewell, Finney, & Jones, 2007; 
National Commission on Accountability in Higher Education, 2005; Shulock & Moore, 
2005, 2007; U.S. Department of Education, 2006). Moreover, several states have recently 
enacted performance funding systems, including Texas and Washington (Blum, 2007; 

Dougherty & Natow, 2008; Southern Regional Education Board, 2007; Washington State 

2 

Board for Community and Technical Colleges, 2007). 

Besides state performance accountability systems, another major approach to 
performance accountability involves the development of voluntary systems of 
accountability by several higher education associations. This strategy was largely 
prompted by the 2006 report of the Spellings Commission (U.S. Department of 
Education, 2006), which called for the development by states and higher education 
institutions, systems, and associations of “outcomes-focused accountability systems 
designed to be accessible and useful for students, policymakers, and the public, as well as 
for internal management and institutional improvement” (p. 24). 

In 2007, the National Association of State Universities and Land-Grant Colleges 
(NASULGC) and the American Association of State Colleges and Universities (AASCU) 
launched the Voluntary System of Accountability (VSA). In this system, participating 
colleges and universities post on their websites a “College Portrait” that contains certain 
standard information: student and family consumer information (student characteristics, 
degree offerings, cost of attendance, living arrangements, rates of graduation and transfer, 
and student post-graduate plans); student experiences and perceptions; and learning 
outcomes (Fischer, 2007; Keller & Hammang, 2008; Miller, 2008). 
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Meanwhile, the American Association of Community Colleges and the Association of 
Community College Trustees, in partnership with the College Board, have launched an 
effort to develop a Voluntary Framework of Accountability (VFA) for Community 
Colleges. Driving this effort has been a concern that the aforementioned VS A and other 
systems of accountability — while useful — are tailored to four-year colleges and may 
not be wholly applicable to community colleges. In fact, the uncritical use of such 
measures might lead to “misperceptions of community colleges and an underestimation 
of their effectiveness and contributions” (American Association of Community Colleges, 
Association of Community College Trustees, & College Board, 2009, p. 1). 

The Community College Research Center (CCRC) study presented here, commissioned 
by the College Board, has collected data to inform the development of the Voluntary 
Framework of Accountability for Community Colleges. CCRC researchers identified the 
performance indicators that states are already using for their community colleges and 
interviewed state higher education officials and local community college leaders on their 
experiences with the collection and use of such performance data. More specifically, 
CCRC examined: 

• The content of state performance accountability systems, focusing on what 
performance indicators are most common across states. 

• The extent of the compatibility of those state performance data with the data 
required by the Integrated Postsecondary Education Data System (IPEDS) and the 
regional accreditation associations. 

• The ways that the states make public the performance data they collect on 
community colleges and how community colleges are compared to each other. 

• The ways that the performance data are collected by states and the ways that state 
and local community college officials assess the data collection process. 

• The ways that performance data are used by state and local community college 
officials in decision making and the ways that each assesses the data usage 
process. 



This report lays out our findings on these questions and provides recommendations for 
the content of the Voluntary Framework of Accountability for Community Colleges. In 
particular, the report makes recommendations regarding which performance indicators to 
consider, how to better measure indicators, ways to make the collection of performance 
data easier for community colleges, and ways to encourage better use of performance 
data. 
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RESEARCH METHODS 



The data presented in this report are based on a survey of state higher education officials 
and local community college leaders in 10 states, conducted in May and June 2009. 
CCRC researchers selected the states in order to maximize variation across two criteria: 
type of performance accountability and region of the country (see Table 1 below). We 
included 5 states that have performance reporting alone and 5 that combine performance 
reporting with performance funding. Both performance systems involve the reporting of 
data on institutional performance to the state, but in the case of performance funding, the 
data affect state appropriations to community colleges. A small portion of the state 
appropriations to the community colleges — somewhere between 1 and 5 percent — is 
tied to institutional performance on specific indicators (Burke & Associates, 2002; 
Dougherty & Hong, 2006). Region of the country was used as a sampling dimension in 
order to pick up differences in state political culture and socio-political conditions. 

Table 1. 

States Analyzed 



Region 


States with Only 
Performance Reporting 


States with Performance 
Funding As Well As 
Performance Reporting 


New England 


Massachusetts 




Mid Atlantic 


Maryland 




Southeast 




Florida, North Carolina 


North Central 


Illinois 


Ohio 


South Central 




Louisiana [PF not funded] 


Plains 


Texas 




Northwest 




Oregon [PF suspended] 


Southwest 


California 




Note: In 2007, Texas established performance funding for its universities but not its 
community colleges. Hence, we have categorized it under performance reporting. 



Data gathering consisted primarily of interviews with state higher education officials and 
local community college leaders and a search of publications by state agencies and 
national higher education policy organizations. Within each state we aimed to interview a 
state official who has main oversight over the performance accountability system for 
community colleges and two local community college officials, one at a large college and 
one at a small one. The purpose for interviewing two local officials was to shed light on 
how the experiences of local community college officials in generating and using 
performance accountability data differ by size of community college. 

We got a very good rate of response except in the case of Louisiana and Illinois. Due to 
the very tight time line for this report, we were not able to get data back from them in 
time. However, data on their performance accountability systems are included in Table 2 
in the Findings section below and in the Appendix. 
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FINDINGS 



Our analysis focuses on the content of state performance accountability systems for 
community colleges, their degree of fit with data demanded by IPEDS and the regional 
accrediting associations, the way that state performance data are collected and reported, 
the uses to which state and local officials put those data, and the difficulties that they 
encounter in collecting and using performance data. Its aim is to inform the design of the 
proposed Voluntary Framework of Accountability for Community Colleges. 

Indicators Used by State Performance Systems 

The study’s 10 states collect a vast number of indicators. By indicators we mean specific 
characteristics of a community college that are deemed of interest, such as its enrollment 
of certain types of students or the number of students it graduates. 4 The 10 states together 
collect about 140 indicators and CCRC researchers grouped them into various categories 
(see the list in the Appendix). The indicators include input characteristics such as 
enrollments, process characteristics such as extent of vocational programming, and output 
characteristics such as graduation rates. 

Focusing on the indicators that are considered in at least 3 of the 10 states, we find that 
states commonly have input indicators involving total enrollments; types of populations 
served (for example, the proportion of high school graduates enrolled); and enrollments 
disaggregated by some student characteristics. 5 The most common breakdown is by 
race/ethnicity, with breakdowns by income, gender, age, and immigration status being 
much less common. 

With regard to process indicators , the most common are those concerning tuition and 
fees; expenditures (particularly on instruction); faculty characteristics; and the extent of 
vocational, remedial, dual-enrollment, and online programming. The measures for 
curricular programming primarily comprise enrollments in particular kinds of programs. 
Interestingly, only one of the 10 states has a measure for transfer-education 
programming, despite the importance of preparation for transfer to university 
baccalaureate programs as a community college mission. 

Finally, with regard to output indicators , the most common are rates of remedial and 
developmental education success, graduation from the community college, transfer to a 
four-year college or university, and employment outcomes. They are followed by other 
indicators, such as attainment of a GED or other such degree for high school dropouts 
attending community college; the number of students earning a certain number of credits; 
the number of students passing certain specific courses, such as college-level math; 
retention and persistence rates; after-transfer performance; passing licensure exams; and 
student and employer satisfaction with community college services. 
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Table 2. 

Common Indicators in State Performance Systems 



Indicator „ 

Stat< 

Enrollments 6 

Population served 4 

Enrollments broken down by race/ethnicity 8 

Enrollments broken down by income (primarily measure of financial aid receipt) 6 

Enrollments broken down by gender 3 

Enrollments broken down by age 1 

Enrollment broken down by language status 1 

Process Indicators 

Tuition and fees 6 

Expenditures 4 

Faculty characteristics 3 

Vocational programming (enrollments, clients served, offerings) 5 

Remedial or developmental education programming (enrollments) 4 

On-line programming 4 

Dual-enrollment programming (enrollments) 5 

Output/Outcome Indicators 

High school completion 4 

Remedial and developmental success 8 

Credits earned 4 

Specific courses passed (e.g. college level English or math) 3 

Retention 7 

Graduation from community college 10 

Transfer to four-year colleges and universities 9 

After-transfer performance 5 

Employment 7 

Passing licensure exams 7 

Student satisfaction 3 

Employer satisfaction 4 

Disaggregation of Outcome Indicators 

Success broken down by race/ethnicity 4 

Success broken down by income 1 

Success broken down by gender 2 

Success broken down by age 2 



Note: For more detail, see the Appendix. 

It is noteworthy how rarely states break down their output/outcome indicators by student 
background, despite the many indicators included in the state performance accountability 
systems that we examined. Only 4 of the 10 states have output/outcome indicators broken 
down by race/ethnicity, only one state has an indicator broken down by income, only 2 
break down indicators by gender, and only 2 states by age. 6 
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It is also striking that the state performance indicators do not include more indicators for 
student learning. To be sure, we do find indicators for completion of remedial and 
developmental education (present in 8 states), passage of certain gatekeeper courses such 
as college-level mathematics and English (3 states), or attainment of a certain number of 
credits (4 states). However, these are not indicators for how effectively colleges are 
inculcating general knowledge and skills across the entire student body. Only one state 
(Florida) has had an indicator for general student learning over the course of college, 
measured in terms of passing rates on its College Level Academic Skills Test (CLAST). 7 

Compatibility With IPEDS and Regional Accreditation 

In establishing performance accountability systems, states ask community colleges for a 
lot of data. Collecting this information could be made easier if the state data demands are 
fairly compatible with those put on colleges by the federal government through the 
Integrated Postsecondary Education Data System (IPEDS) (National Center for 
Education Statistics [NCES], 2009); and by the regional accrediting associations, such as 
the Southern Association of Colleges and Schools, the North Central Association of 
Colleges and Schools, or the Western Association of Colleges and Schools. 

When we asked our state respondents for their perception of the degree of fit between 
state performance data demands and those of IPEDS and the regional accreditors, we 
found that they perceived a relatively low degree of overlap in the case of the first and a 
moderate degree in the case of the second. Tables 3 and 4 below summarize their 
responses. The coding into high, medium, and low degree of fit is our own. 

Fit with IPEDS demands 

Most of the data the IPEDS system asks from colleges are not concerned with college 
outputs or outcomes. In terms of outputs, IPEDS only asks for retention, transfer, and 
graduation rates for full-time, first-time students. Otherwise, it asks for data on 
enrollments, financial aid, number of degrees conferred, faculty and staff characteristics, 
and institutional finances (NCES, 2009). Given the many state performance indicators 
addressing institutional outputs, it should not be surprising that, for the most part, our 
respondents do not perceive a high degree of overlap between IPEDS and state 

o 

performance accountability systems, as shown in Table 3 below. 
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Table 3. 

Fit Between State Performance Indicators and IPEDS Data Requirements 



Fit 


Responses from State Officials 


High 




NC 


Unit record data used for both. The state reports to IPEDS from the statewide SUR 
[student unit record] data system. 


Medium 




MA 


For the most part it is the same. The colleges don't report the course completion rate 
to IPEDs or persistence data to IPEDS but they report it to state. But the 
performance accountability data are compatible and consistent with IPEDS. 


TX 


Try to match IPEDS when possible, but the IPEDS definitions change and we make 
sure data is consistent across years. 


MD 


Some overlap. There are some indicators that are not reported to IPEDS. The state 
measures break cohorts into needing developmental ed and taking it, needing 
development ed and not taking it, and college ready, but IPEDS does not track at 
that level of detail. 


Low 




CA 


Some overlap. Some of the descriptive data in the report matches the IPEDS-IC 
reporting but the graduation rates in the [state] ARCC [Accountability Report for 
Community Colleges] report do not match the IPEDS-GRS at all. Vastly different 
methodology. Many other things are collected in IPEDS (HR, finance, etc.) that are 
not covered in ARCC. 


OH 


IPEDS has limited outcomes measures. 


OR 


IPEDS does not fit community colleges. IPEDS is geared more to 4-year institutions 
and traditional students. State CC Board and the community colleges are unhappy 
the CC data are used — they feel it is misleading. 


FL 


Data reported to IPEDS and performance accountability data are completely 
different. 


No 

Response 


LA, IL 



Sources: State higher education officials, sometimes supplemented by local community college 
officials. 

Fit with regional accreditation association demands 

In the last 10 to 20 years, the regional accreditation associations have put greater 
emphasis on institutional performance and have shifted their accountability systems 
accordingly (Biswas, 2006; Terkla, 2001). But do our respondents perceive a close or 
loose fit between the data the accreditors are demanding and the data the state 
government requires? As Table 4 below shows, most of our respondents indicated a 
moderately close fit. Again, the categorization of perceived degree of fit as high, 
medium, or low is our own. Ohio provides an example of a state where the fit between 
the state performance data and the requirements of the regional accreditors appears to be 
in the middle range. As a local community college institutional researcher noted, 
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The fit is reasonably good — though the state level reporting indicators 
are not as rich as is required to truly meet the needs of our accrediting 
body. The primary indicators of success which, to date, are monitored by 
the Regents are as follows: 3-year persistence/graduation/transfer, year-to- 
year persistence, [and] remedial education requirements. These data are 
consistent with the requirements of our accrediting body, and they do 
provide us with a set of comparisons against which we can benchmark. In 
order to increase the success of our students, however, we find it necessary 
to conduct more detailed analyses than are available through this system. 

Table 4. 

Fit Between State Performance Indicators and Regional Accreditation Standards 



Fit 


Responses from State Officials 


High 




FL 


There is a lot of overlap. Institutions use their performance accountability reports 
very frequently while going through accreditation process. 


Medium 




OH 


The fit is reasonably good though the state level reporting indicators are not as rich 
as is required to truly meet the needs of the accrediting body. The primary indicators 
of success monitored by the Regents are consistent with the requirements of the 
accrediting body, and they do provide us with a set of comparisons against which we 
can benchmark. However, in order to increase the success of students, colleges find 
it necessary to conduct more detailed analyses than are available through this system. 


MD 


[Following is the researchers’ analysis]: The Middle States Commission on Higher 
Education (MSCHE) has 14 standards that institutions arc expected to address. The 
Maryland Higher Education Commission (MHEC) Performance Accountability 
Report (PAR) for community colleges has a number of indicators that are directly or 
indirectly related to many of those MSCHE standards. 


OR 


State CC board has been told that the performance measures have come into good 
use for accreditation puiposes. 


NC 


SACS requires colleges to address an issue they deem relevant and critical to student 
learning: a Quality Enhancement Plan (QEP). The accountability measures that are 
evaluated through the accountability process, such as success in developmental 
coursework, can be used for QEP. 


MA 


Many of the state goals are important to accrediting agencies. But accrediting 
agencies look at the data and analyze at a deeper level. They are much more 
interested in specific programs and they have a much more comprehensive view. 


Low 


None 


No 

Response 


LA, IL, TX, CA 



Sources: State higher education officials, occasionally supplemented by local community college 
officials and our analysis of material produced by the states and by the accrediting 
associations. 
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Data Reporting 



The impact of performance data depends in good part on whether the data are publicly 
reported and the way that they are reported. Performance data will have much less impact 
if they are not publicly reported, particularly by institution. At the same time, public 
reporting might lead to a misperception of real institutional performance if the differences 
between colleges in student composition, organizational resources, and conditions in their 
organizational environment are not taken into account. 

Extent of public reporting 

The state performance data are publicly reported in all 10 of the states we examined, with 
the preferred method being publication on the internet. Moreover, the data are usually 
reported to state officials, particularly state legislators, as shown in Table 5 below. 9 

Table 5. 

The Ways that State Performance Data Are Reported 



Category of Data Reporting 


States 


#of 

States 


Publication on the Internet 


TX, CA, LA, OR, MA, MD, IL, FL, NC, 
OH 


10 


Print publication by the State Board 


NC 


1 


Published in institutions’ catalogues 


NC 


1 


Report to the legislature 


TX, OR, MA, MD, FL, NC 


6 


Report to governor’ s office 


MA, MD 


2 


Report available on request 


MD, FL 


2 


Report available in state library 


MA 


1 


Report to institutions 


OR, FL 


2 



Sources: Interviews with state higher education officials. 

Extent and methods of disaggregation of data by individual institution 

All 10 of the states break down the performance data by institution. The logic for 
breaking down results by individual institution rather than just presenting data for the 
whole community college system is explained by an Oregon local community college 
official: 
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[In] previous legislative sessions, key performance indicators were not 
detailed by community colleges — they were for all community colleges 
in the state. Individual colleges never saw their own data relative to how 
the state performed; therefore there was not much interest at the individual 
level. . . .[Then] the individual key performance measures were detailed by 
college. . .At that point on [they] took on deeper meaning and importance. 

There wasn’t the same feeling of accountability when there were no 
benchmarks. 

However, as Table 6 below shows, the states differ in how they treat the differences 
among their colleges. Many have argued that performance data for community colleges 
need to be corrected to take into account differences, such as in the proportion of 
disadvantaged students they enroll. Otherwise, many community colleges could appear to 
be performing badly, despite the fact that they may be actually doing comparatively well 
on a value added basis (Dougherty & Hong, 2006; Dowd & Tong, 2007; Erisman & Gao, 
2006). To address this issue, some states, such as California and Texas, group the 
colleges by peer groups. California’s system is particularly sophisticated in that it 
identifies different peer groups for each of the seven performance measures on which it is 
comparing the colleges. Hence, each college has seven different peer groups, one for each 
measure. The peer groups are created through cluster analysis, a statistical procedure to 
create groups on the basis of certain variables (California Community College, 2009). 10 

Table 6. 

Adjustments Made to Account for Differences in Institutional Capacity 



Type of Adjustment 


States 


#of 

States 


Institutions are grouped with their peers 


CA, TX, MA 


3 


Adjustments are made to financial data on the basis of institutional 
characteristics 


FL 


1 


No adjustments made 


OR, MD, NC, 
OH 


4 


No Response 


LA, IL 


2 



Sources: Interviews with state higher education officials, supplemented by state documents. 

The Data Collection Process 

Performance data are principally gathered by the state through the regular reports that the 
community colleges file with it on their enrollments, finances, and other characteristics. 
These reports are supplemented by additional reports submitted by the community 
colleges that provide information from surveys they conduct of employer satisfaction and 
their graduates’ satisfaction and current circumstances, and data they gather on licensure 
pass rates (although the state sometimes secures these data on its own) and transfers to 
private colleges (the state usually gathers the data on transfer to public institutions). 
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How well this process works from the perspective of state higher education officials on 
the one hand and local community college leaders on the other is discussed next. 

State officials’ assessment of the data collection process 

Only a few of the state officials we interviewed reported difficulties in securing 
performance data from the colleges and their complaints were not strong. They focused 
on difficulties in getting high quality data. A likely reason for this low rate of complaints 
is that, as noted above, the vast bulk of performance data are collected as part of the now 
highly routinized data collection that states conduct for enrollment, financial, and other 
information. 



Table 7. 

Experiences of State Officials in Securing Performance Data 



Type and Extent of Difficulties 


States 

Reporting 


#of 

States 


No Difficulties 


OR, MD 


2 


Difficulties in Getting High Quality Data 




2 


Challenges include typical hindrances to centralized data collection such 
as different business practices and data capacities across the campuses. It 
is also difficult to initiate the collection of new data fields and to revise 
existing data fields. 


MA 




Usually, the difficulties are related to the quality of data at the campus 
level. State has no problem getting the data. As long as colleges submit 
clean data and get the data in on time, the state is able to produce high 
quality reports. 


FL 




Other Difficulties 




1 


Delays in receiving data from some outside agencies and then mapping it 
to the unit records. This is not a problem from the colleges as annual 
reporting plan requires compliance or they hold the president’s salary. 


NC 




Cannot Identify Difficulties Yet 

(performance funding system just beginning) 


OH 


1 


No Response 


LA, IL, 
TX 


3 



Sources: Interviews with state higher education officials. 

Local college officials’ assessments of the data collection process 

But if state officials appear largely happy with the data collection process, local 
community college officials have a more critical assessment. The most common problem 
pointed out by local college officials with regard to collecting and reporting data to the 
state concerns the difficulties of acquiring certain data that the state demands, particularly 
information originating from surveys of employers, graduates of the colleges, and non- 
returning students (see Table 8 below). It is hard to get a high response rate on those 
surveys. In addition, it is sometimes difficult to get “third party” data in the form of 
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licensure pass rates or number of transfers to private colleges and universities. Other 
complaints made with less frequency are that it is hard to code certain state data and that 
there are not enough trained staff at the community college to properly collect the 
required data. 

Our local respondents sometimes indicated that the size of a college made a difference in 
how easily a college could report data to the state. However, it is interesting that 
sometimes it was big size and sometimes it was small size that was an impediment to data 
collection (see Table 9 below). For small colleges, the hindrances lie in lack of 
organizational resources for institutional research. For large colleges, the impediments lie 
in how to capture in state data categories the complexity of the colleges’ operations. One 
of our local respondents in North Carolina captured the countervailing effects of size on 
the requirements of collecting and reporting data to the state: 

Larger colleges often have superior resources in terms of sophistication 
and number of IR staff (though this can vary with the interest of the 
President in having the capability). On the other hand, simplistic data 
definitions can sometimes work against large colleges. Also, given the size 
of large college databases, more sophisticated data handling methods are 
necessary. (Some small colleges can literally do manual manipulations 
that are impossible for large colleges to handle.) 

Table 8. 

Experiences of Local Officials in Collecting and Reporting Performance Data 



Experience 


States 


#of 

States 


Few or No Problems 




1 


Collecting and reporting performance data are relatively easy since the measures 
are based on previously required data submitted each term or annually to the state 
system office. 


CA 




State Measures Are Hard to Collect 


MD, CA, 
NC, OR 


4 


The most difficult data to gather arc the indicators that require the use of alumni 
surveys and employer surveys. Response rates on these instruments are typically 
very low, making it very difficult to draw any useful conclusions from these data. 


MD 




The validity and reliability of data gathered from alumni and employer surveys arc 
extremely questionable. 


CA 




The employer satisfaction item is another “ought to” measure. However, the nature 
of firms (employ many occupations that are not specific to the firm’ s industry) and 
the fact that single employers may employ students from several of the colleges 
makes mass surveying impossible. Employers are also over-surveyed by all kinds 
of agencies. 


NC 
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Table 8. 

Experiences of Local Officials in Collecting and Reporting Performance Data 



Experience 


States 


#of 

States 


The most difficult area is gathering non-returning student data.. ..the response rates 
are not high enough to make the reports valid. Employer data is also hard to collect 
in a comprehensive way. Reasons why this is difficult include the response rate 
problem, finding the right person to respond for the business, and getting general 
data from an employer who may have both successful and unsuccessful former 
students as employees. 


NC 




It is often difficult to receive completed surveys from the students. A prepaid, self- 
address envelope is provided; and it usually requires fracking returned surveys and 
requesting a 2nd, 3rd and even a 4th mailing for students who have not responded 
in order to meet a 50% response rate or better. 


NC 




Some of the data required for the Performance Measures are difficult to track, such 
as the number of students who transfer to a four-year college or university. 
Students transferring to one of the sixteen (16) public state universities are tracked 
by the state, but students transferring to a private college or university must be 
tracked by each community college and reported to the System office if they are to 
be measured as paid of the Performance of College Transfer Students. 


NC 




Some 3rd-party data (licensing boards, employment department) [are] difficult to 
acquire. 


OR 




Lack of access to data sources such as university and private college data. 


NC 




Hard to Code State Data 




2 


There is always a difference in definitions in regard to how the state might define 
retention or success rate vs. institutions. From an institutional effectiveness 
standpoint they look at more detailed measures, where the state is looking at the 
big picture. 


OR 




Biggest problem arises as there is no agreed upon system for coding of 
coursework, especially for basic skills courses, in English, math and ESL. There is 
no uniform coding system statewide or district wide. Even among the 9 colleges in 
LA, there is no agreement.... It is therefore difficult currently to measure and 
analyze basic skills advancement and completion. There will be an effort this year 
to recode all the courses below freshmen level at all 1 10 colleges statewide. 


CA 




Lack of Enough Trained Staff to Accurately Report Data 


MA, OR 


2 


Data collection — staff needs more training especially in data entry. There are 
more offices entering data in the DataTel system now than before. Data entry is 
setting comnlicated so more training is needed. 


MA 




Many of the colleges are understaffed; IT doesn’t have the programming capacity 
to collect the data. 


OR 




Date by Which Data Are Required to Be Submitted Is Too Early 


MD 




No Response 


LA, IL 





Sources: Interviews with local community college officials. 
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Table 9. 

Perceived Impacts of College Size on Collecting and Reporting Data for State 



Type of Problem 


States 


#of 

States 


Small College Problems 






Fewer Institutional Research Resources 




5 


Smaller colleges have fewer resources to devote to institutional research, so 
perhaps a smaller college would rely more on statewide reports without the 
benefit of conducting local research and analysis to dig deeper into specific 
areas. 


CA 




Smaller colleges can be expected to have fewer staff resources and perhaps 
weaker technology, so they probably have more trouble producing their own 
performance data than larger colleges like us. 


FL 




The management information systems used by smaller institutions are usually 
not as powerful and lack the capabilities of systems used by larger 
institutions. 


MD 




Some smaller colleges are hindered by limited IT infrastructure which may 
make data extraction and retrieval more difficult than for their large-college 
colleagues. 


MD 




Larger colleges often have superior resources in terms of sophistication and 
number of IR staff (though this can vary with the interest of the president in 
having the capability). 


NC 




Big College Problems 






More Complicated Data to Report 




3 


Simplistic data definitions can sometimes work against large colleges. Also, 
given the size of large college databases, more sophisticated data handling 
methods are necessary. 


NC 




Biggest problem arises as there is no agreed upon system for coding of 
coursework, especially for basic skills courses, in English, math and ESL.... 
There is a bigger sensitivity to this issue of coding at the larger districts - 
those with only one school would not notice the problems with course coding. 


CA 




Harder to get consensus on data. 


MA 




Larger colleges do have more difficulty in meeting the response rate. 


NC 




No Response 


LA, IL 


2 



Sources: Interviews with local community college officials in 10 states. 
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Use of Community College Data at the State Level 

Given the effort and expense involved in collecting performance data, it is important to 
know how extensively state officials use these data and whether they encounter 
significant problems with their analysis. 

Extent of data use 

One of the questions we asked state officials was what decisions are affected by the data 
they collected on the community colleges. For the most part, they stated that the data did 
not much affect state decisions. In fact, only one state indicated that the data were used to 
trigger policy reviews or changes. In Massachusetts, a state higher education official 
noted: 



Student success indicators..., indicators related to the contribution to the 
workforce. . ., and indicators related to higher education access. . .are all 
scrutinized by Board members, Department staff, and policy makers. In 
recent years issues related to these various indicators have led the Board to 
launch taskforces around state and community college graduations, 
transfer policies, and student financial aid. Taskforce outcomes have 
included revised transfer incentive policies, requests to the legislature for 
more financial aid support, new indicators for tracking student success, 
and new programs aimed to increase nursing and Science, Technology, 
Engineering, and Mathematics (STEM) degrees. 

Instead, the most common pattern was one of lack of state scrutiny of and comment on 
performance data and little use of those data to craft state policy. A state official in 
California noted that it was difficult to get the legislature to use the state performance 
data. Local community college officials in Maryland and Florida also noted a lack of 
state use of performance data. In Maryland, a local community college official stated: 

Currently, there is not enough attention paid at the state and college level 
to these data — collectively or individually for the community colleges. 

We submit our reports and occasionally are questioned about progress 
toward a target (“benchmark”), but these queries are relatively benign. 

An official at a Florida community college added: “If data [are] not utilized, it would be 
helpful not to have to report it.” 11 

Assessment of data usage at the state level 

Interestingly, only 2 states reported difficulties in analyzing the performance data they 
received from community colleges (see Table 10 below). This largely positive assessment 
could reflect the quality of those data but it may also reflect that the states are using the 
performance data primarily for compliance purposes rather than institutional diagnosis. 
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Table 10. 

Experiences of State Officials in Using Performance Data 



Type of Problem 


States 


#of 

States 


No Major Problems 




4 


The process works fairly well as long as there is money. Otherwise there is usually 
no problem with data collection, analyses, creating reports etc. There is a 
committee with representatives from CCs and the members do their best when a 
problem occurs. . . . As long as colleges submit clean data and get the data in on 
time, the state is able to produce high quality reports. 


FL 




There have been no substantial problems. Since the adoption of the current system, 
the Department has revised indicators, added new indicators, and altered the report 
format. All of this was done in collaboration with the Performance Measurement 
Taskforce to ensure campuses had sufficient input on the process. 


MA 




No problems at all. PR system has been working well. 


MD 




No problem reported 


NC 




Lack of Sufficient Data Analysis Resources 




1 


The funding shortage doesn’t just occur to the CCs. Representatives at the state 
level who work with the data are grossly understaffed. They do not have the 
resources to provide the level of support to make this more effective [from local 
CC official]. 


OR 




Results are Varied and Difficult to Tie to any Causal Factor 


CA 


1 


Cannot Identify Difficulties Yet (performance funding system just beginning) 


OH 


1 


No Response 


LA, IL, 
TX 


3 



Sources: Interviews with state higher education officials. 

Data Use at the Uocal Uevel 

Particularly because the demands of data acquisition fall primarily on them, it is 
important to determine how useful community colleges find the data they report to the 
state governments and what suggestions they have for improving those data. 

Extent of data use 

Most of the local community college officials we interviewed stated that they do utilize 
the data they report to the state in making local decisions. The modal pattern is one of 
heavier data use than is the case with state officials, though there is considerable variation 
in the extent of local use. 12 

The main areas of influence are instructional and student support policies affecting 
student retention and graduation, planning and budget decisions, and less so, human 
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resource policy (see Table 11 below). For example, an official of a Texas community 
college noted: 

The performance indicators at [the college] drive programmatic and 
curriculum changes as well as student support services. Data showing 
trends in gender, ethnicity, success, retention, financial aid, student age, 
and licensure pass rates have all contributed to decisions made regarding 
. . .community outreach programs. . . .Procedures in enrollment 
management, certificate and degree completion, recruitment, retention, 
and success initiatives are based on these data. The [college’s] Minority 
Male Initiative also based its direction and benchmarks on these data 
showing the trends in African American and Hispanic male enrollment 
and completions. The [college] Strategic Plan refers to these performance 
indicators to create a baseline of current employee demographics to 
project the recruitment and retention of faculty and administrators. 

Table 11. 

Local Decisions That Performance Indicators Affect 



Effect of Indicator 


States 


#of 

States 


Student Success (Instruction and Student Support) 
Decisions 




6 


Efforts to improve student success (e.g. retention and 
graduation) 


MD, OH, CA, NC, TX, 
MA 




Efforts to improve student engagement 


MD 




Minority enrollment and completion 


TX, CA 




Support services for allied health students to pass 
licensure exams 


NC 




Improvements to academic standards 


MD 




Planning and Budget Decisions 




6 


Planning and strategy 


CA, TX, MD, NC 




Resource allocation and budgeting 


MD, MA, OR 




Human Resource (Faculty and Staff Policy) Decisions 




3 


Recruitment of faculty and administrators 


MD, TX 




Enhance hiring of minority employees 


MD, TX 




Faculty and staff compensation (bonus funds) 


NC 




No Report of Local Data Use 


FL 


1 


No Response 


IL, LA 


2 



Sources: Interviews with local community college officials. 

At the same time, we must not overestimate the impact of the state data. Several colleges 
reported that they do not rely very much on the state performance data but instead rely 
much more on their own internally generated performance data. A local community 
college official in Ohio noted: 
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Data provided to the Board of Regents are provided mainly for 
compliance purposes. The college generates a large amount of data to 
answer questions pertaining to student access and success. Data provided 
to the Regents do play some part in this analysis. However, the institution 
makes use of a wide range of data in addition to those provided to the 
Regents. 

An official of a North Carolina community college added: 

State system data were initially less than useful for improvement 
purposes. . . .State requirements do not include information based on 
cohorts, thus can be misleading. Colleges with their own internal data 
warehouses. . .are able to do much more accurate data mining to find the 
story behind the performance measures, and thus to develop 
improvements. 

As the last remark indicates, part of the reason that local colleges do not rely on state data 
as much as we would expect is due to problems with the content of those data. Let’s now 
turn to local officials’ assessment of the factors affecting their degree of usage of state 
data. 

Assessment of data use at the local level 

Our local respondents noted a number of impediments to their effective use of state 
performance data. As Table 12 below indicates, the most common complaints were that 
the state data were not always timely, they did not give contextual causes of performance 
differences, there were issues of measurement reliability, colleges lacked analytic 
capacity, and they lacked the necessary commitment to a culture of evidence. 

The criticism about timeliness of data turned on the fact that by the time the state reported 
the data the students involved had moved on and circumstances had changed. Those 
noting problems with contextual causes contended that the state data did not take into 
account geographical, financial, and other factors that might help explain seeming 
differences in performance. 

Measurement reliability was an issue particularly for local community college officials 
from small colleges who were concerned that state data definitions (such as focusing on 
first-time, full-time students) resulted in very small samples at small colleges. As a result, 
the findings for those colleges were unreliable because a shift in only a couple of cases 
could greatly skew the data. 

Local community college officials also turned inward in explaining limits to their 
analysis of performance data. They noted that colleges not infrequently lacked enough 
staff with the requisite skills to adequately analyze the data. Moreover, they pointed to a 
lack of a strong culture of evidence or inquiry that would support the in-depth analysis of 
data for purposes of informing practice. On this last point, a respondent from North 
Carolina noted: “It takes a lot of inclusion in the process, ownership of the questions to 
be answered, training in using data to improve, and assurance that actions will be taken, 
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and resources will be available, to help overcome the issues that result in poor 
performance.” 



Table 12. 

Main Local Problems with Utilizing State Performance Data 



Problem 


States 


#of 

States 


State Is Not Collecting Important Data 




2 


Lack of comprehensive data on employment outcomes for leavers and 
graduates and current students. . ..The changes I would make for the state 
would be to link relevant information from other state databases 
(Unemployment wage records; state tax returns. . .) so that community 
colleges can receive credible, comprehensive data on how effectively we’re 
developing the regional workforce. 


MA 




I would like to see additional information from the University System about 
students who failed there, returned home and enrolled in [our college], and 
returned later to the University. These students are currently not included in 
performance measures and they are often turn-around stories. 


NC 




Collected Data Are Not Always Timely 




3 


Some data is “old” by the time it is incorporated into the reports — cohort 
tracking data, for example, of necessity is reporting on students who began 
four and five years ago. ...It is a struggle to convert some performance report 
data into courses of action because by the time the data are analyzed, the 
students on which the data are examined have already gone through the 
system. 


MD 




The exact methods the state uses to calculate performance is usually a year or 
two later than we can produce ourselves. Therefore, I would say the biggest 
issue is the lag time to obtain state reports and benchmarks that arc too old 
compared to the more timely measures we can produce on our own 
performance. 


FL 




You have a new group with different goals every year but your data is not 
up-to date (2 years old) and it is being a problem. 


MA 




State Data Do Not Give Contextual Causes of Performance Differences 




4 


Within the community college sector, performance rates do differ 
significantly. A weakness of these [state] reports is that they do not contain 
any data that might be used to help explain differences in these rates across 
institutions within the same sector. 


OH 




Perhaps some index of the at-risk population should be used to weight data. 


NC 




State measures don’t incorporate local factors (geographic location, finances, 
etc.). Example of transfer rates. [Our college] has higher transfer rates than 
other Oregon CCs because they are near many 4-year colleges, but many of 
their students often transfer before they have received their award, which 
pulls down their graduation rate. 


OR 
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Table 12. 

Main Local Problems with Utilizing State Performance Data 



Problem 


States 


#of 

States 


It does not address the local variations. . ..The ARCC is too macro to be 
useful at the local level. By adding a peer grouping reporting, some of these 
problems have been mitigated, but again, these data become less useful at the 
local level. 


CA 




Measurement Reliability Issues 




4 


This college is very small and has a primarily non-traditional student 
population. Therefore, state cohorts that are based on fulltime degree- 
seeking, first time in college students result in very small numbers for the 
college. 


FL 




Our college is very small and given a small “N [number of respondents],” 
percentage figures are more volatile than for larger populations. 


OR 




The number of EMT [Emergency Medical Technician] and/or BLET [Basic 
Law Enforcement Training] students testing is low so just one failure affects 
the pass rate percentage. 


NC 




One area in which the data warehouse has been a problem is graduation rates. 
The programming has left out a group of students that accounted for 15-20% 
of our graduates. It's important for IR people at the college level to have 
access to programming details so that they can see where the errors might be, 
and that the programmers listen to the IR people and make the corrections in 
a timely fashion. 


NC 




The success or lack of success in basic skills is skewed by the lack of 
common measures in such things as different cut scores and different 
placement testing instruments, and colleges and faculty arc free to set 
multiple measures. A common measure or assessment requirement would 
face resistance from the faculty. 


CA 




Inadequate Resources for Analysis 




3 


Superficial analysis, often because the analytic capability and/or access to 
additional data elements is restricted. 


NC 




Budget cuts mean colleges may be understaffed. Also the lack of funding 
limits the IT resources to improve things. In Oregon, very few CCs have 
fully developed degree audit systems whereas other states might. 


OR 




The changes I would like to see in the [state] data warehouse would involve 
making more of the data we submit accessible through the warehouse. ... I 
would like to have either more drop-down menus or else greater expertise 
(and possibly greater access) with the [state] warehouse. 


MA 
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Table 12. 

Main Local Problems with Utilizing State Performance Data 



Problem 


States 


#of 

States 


Lack of Culture of Evidence or Inquiry 




3 


We have work to do at our own institution to make the report and the data 
more regularly used in planning, budgeting, and decision-making. 


MD 




Resistance of faculty, staff, and administrators where there is not a culture 
that permits self-examination. It takes a lot of inclusion in the process, 
ownership of the questions to be answered, training in using data to improve, 
and assurance that actions will be taken, and resources will be available, to 
help overcome the issues that result in poor performance. Rewards help. 


NC 




Some institutions are dead serious about compliance reporting, generating 
(and using) timely, accurate data. Some institutions have other values and 
priorities. 


MA 




No Response 


LA, IL 


2 



Sources: Interviews with local community college officials in 8 states. 
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CONCLUSIONS AND RECOMMENDATIONS 



Below we summarize the principal findings from the study, commissioned by the College 
Board and conducted by the Community College Research Center (CCRC), to collect 
data useful in the development of the Voluntary Framework of Accountability (VFA) for 
Community Colleges. We also provide recommendations for the elements that need to be 
considered when designing a performance accountability system for community colleges. 

Principal Findings 

Commonly used indicators 

The 10 states in CCRC’s study together collected data on some 140 indicators to assess 
their performance (see the Appendix). If we focus on the indicators that at least 3 of the 
10 states use, the most common input indicators concern total enrollments, proportion of 
population served, and enrollments broken down by race/ethnicity and income. Only 3 
states have enrollment indicators broken down by gender, only one has an indicator 
broken down by age, and only one has an indicator broken down by language status 
(enrollment in English as a Second Language courses). 

The process indicators used in 3 or more states are those involving tuition and fees; 
expenditures (particularly on instruction); faculty characteristics; and extent of 
vocational, remedial, dual-enrollment, and on-line programming (largely measured in 
terms of enrollments). Only one state has an indicator for the extent of transfer- 
preparation course programming. 

Finally, with regard to output/outcome indicators , the most common are rates of 
graduation from the community college, transfer to a four-year college or university, and 
remedial and development success. They are followed by other indicators such as high 
school completion, the number of students earning a certain number of credits, the 
number of students who pass certain courses such as college level math, retention and 
persistence, after-transfer performance, employment outcomes, passing licensure exams, 
and student and employer satisfaction with community college services. 

It is striking that student learning does not get more attention despite the many different 
output/outcome indicators that populate state performance accountability systems in the 
10 states surveyed. There are indicators that touch on learning but it is in the rather 
indirect form of remediation completion, enrollment in certain gatekeeper courses such as 
college-level mathematics and English, or attainment of a certain number of credits. Only 
one state (Florida) has had an indicator that deals more generally with learning during the 
college years, in the form of pass rates on a rising junior exam. 

Moreover, we have noted the relative dearth of indicators broken down by student 
background: race/ethnicity, income, gender, or age. This inattention is particularly 
pronounced in the case of income, gender, and age and for output/outcome indicators. We 
expand on this point below in our recommendations. 

22 




Compatibility with IPEDS and regional accreditation demands 



We found that our state respondents perceived a relatively low fit between the data 
demands of state performance accountability systems and IPEDS. However, they 
perceived a stronger fit with the data demands of regional accreditors. Given the many 
state performance indicators addressing institutional outputs, it should not be surprising 
that our respondents generally do not see a high degree of overlap with IPEDS, which 
only asks for retention, transfer, and graduation rates. 

Data reporting 

The impact of performance data depends in large part on whether and how they are 
publicly reported. The state performance data are publicly reported in all 10 of the states, 
with publication on the internet as the preferred dissemination method. All 10 states 
break down the performance data by institution, but they differ in how they treat the 
resulting differences among colleges. Some states group the colleges by peer groups 
based on college characteristics (e.g., size, student composition, mission) and, sometimes, 
community characteristics (e.g., area median household income, poverty rate), but most 
of the states do not contextualize the data. 

Data collection issues 

The state performance data are principally gathered by the state through the regular 
reports that the community colleges file with the state on their enrollments, finances, and 
other characteristics. Only a few of the state officials we interviewed reported difficulties 
in securing performance data from the colleges. Their complaints were not strong and 
they focused on difficulties in getting high quality data. However, local community 
college officials have a more jaundiced assessment, focused on the difficulties of 
acquiring certain data that the state demands, particularly data collected from surveys of 
employers, graduates of the colleges, or non-returning students. Our local respondents 
sometimes indicated that the size of a college made a difference in how easily a college 
could report data to the state, with small size and big size bringing different 
disadvantages. 

Use of community college performance data at the state and local levels 

For the most part, state higher education officials observed that the performance data did 
not significantly affect state decisions. Instead, as local community college officials 
noted, the most common pattern was characterized by a lack of substantial scrutiny of 
and comment on performance data by state officials. This lack of scrutiny did not seem to 
stem from the perception of state officials that the state performance accountability data 
were bad. Only 2 states reported difficulties in analyzing the performance data received 
from community colleges. 

Most of the local community college officials we interviewed stated that they do use the 
data they report to the state in making local decisions. The main decisions influenced by 
the data involve instructional and student support policies that affect student retention and 
graduation; planning and budget decisions; and, less so, human resource policy. At the 
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same time, some colleges reported that they do not rely all that much on the state 
performance data but instead rely much more on their own internally generated 
performance data. Our local respondents noted a number of impediments to their 
effective use of state performance data. The most common complaints were that the state 
data are not always timely, they do not give contextual causes of performance 
differences, there are issues of measurement reliability, colleges lack analytic capacity, 
and they have an insufficient commitment to a culture of evidence. 

Recommendations 

The findings from CCRC’s study have definite implications for efforts to construct a 
Voluntary Framework for Accountability for Community Colleges. They fall into four 
areas: performance indicators that should be considered, ways to better measure 
indicators, ways to make collection of performance data easier for community colleges, 
and ways to encourage community colleges and state officials to make better use of the 
data. 

Performance indicators to consider 

Any indicator system should have a balanced combination of input, process, and 
output/outcome indicators. The input indicators should provide incentives for maintaining 
a focus on enrolling less advantaged students. The process indicators should reward 
efforts to be accessible and to provide curricular and other programming that prepares 
students broadly and deeply. And the output/outcome indicators should stimulate colleges 
to focus on socially important outcomes for all students. These indicators applied to 
individual colleges should be balanced with systemic indicators of the performance of the 
community college sector and the higher education system as a whole. 

Input indicators. States need to include indicators for community college access for 
students of different backgrounds. As noted above, the performance accountability 
systems in the 10 states we examined often do not include indicators for access for 
disadvantaged students, including racial minorities, recent immigrants, low-income 
individuals, or women. This finding is significant, because the presence of such 
indicators can help counterbalance the incentive that graduation and job placement 
indicators create for colleges to focus on taking in more advantaged students since they 
are more likely to graduate (Dougherty & Hong, 2006; Erisman & Gao, 2006). 14 

In paying attention to income, an indicator system must not simply focus on whether or 
not students are receiving financial aid. Such information is important but performance 
systems should also look at how access varies by different income levels. Collecting 
information on students’ income is difficult and characterized by high rates of 
nonresponse and misreporting. However, a useful proxy is the average income of a 
student’s neighborhood (more specifically, census tract), which can be simply determined 
by zip code. 

Indicators pertaining to age should focus particularly on access for part-time, non-degree 
seeking adults who are coming to higher education for job retraining or other mid-life 
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course corrections (Erisman & Gao, 2006). Including such indicators in performance 
accountability systems has particular importance today — a time of high unemployment 
and economic restructuring that is likely to go on for a considerable time (Bartels, 2008). 

Process indicators. Four kinds of process indicators are worthy of attention. One type 
measures how accessible community colleges make themselves to less advantaged 
students. At the very least this should include an indicator for how community college 
tuition and fees compare with the median household or family income in the college’s 
district (Carey & Aldeman, 2008). 

A second type of process of indicator should address whether community colleges offer a 
broad range of programs addressed to a variety of student needs and interests. These 
needs range from high school completion (GED, etc.) and remedial education to 
preparation for transfer and workforce development. Our 10 states offer a number of 
different examples of these indicators but one is quite undeveloped: transfer preparation. 
Given the increasingly important role that community colleges are playing in access to 
the baccalaureate in recent years, the development of good measures for the extent and 
quality of transfer preparation is important. One such measure could be the proportion of 
transfer students who, prior to transfer, complete a specified range of courses, such as a 
general education core that the state may have mandated should be given guaranteed 
transfer credit (Dougherty & Reid, 2007). 

But extensive programming can be vitiated if it is not of high quality. Hence, the third 
process indicator that should be given close attention is provision of quality learning. It 
can be indexed by such things as degree of student academic engagement (Carey & 
Aldeman, 2008). The measures in the Community College Survey of Student 
Engagement (2008) provide a useful way of assessing degree of student academic 
engagement. 

Finally, an important process indicator is the availability of resources for community 
colleges (Erisman & Lao, 2006). How well are colleges funded on an FTE basis, 
particularly taking into account the needs of their student bodies, which are less 
advantaged on average than those of four-year colleges? 

Output/outcome indicators. The main necessary elements of a community college’s 
curriculum and programming should be matched by output or outcome indicators that 
measure their ultimate effectiveness. A comprehensive system of indicators needs to 
address eventual success as measured by graduation or transfer and job placement. 

However, for community colleges it is particularly important that summary indicators of 
community colleges outcomes — such as total number of students completing — include 
not just graduates but also those transferring. Just focusing on graduates will miss the 
many community college students who benefited from their time there but are 
transferring without a degree (Dougherty & Hong, 2006; Jobs for the Future, 2008). 

In addition, colleges need to be rewarded for how well they are doing on intermediate 
outcomes that not only are under greater control by colleges but also presage future 
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student success, such as achieving gains in remedial education and adult basic education, 
passing important gatekeeper courses such as college-level English and mathematics, or 
earning college credits at a certain rate (Jobs for the Future, 2008; Leinbach & Jenkins, 
2008; Moore, Shulock, & Offenstein, 2009; Prince & Jenkins, 2005). Analyzing 
performance on intermediate outcomes can be very helpful to colleges to better 
understand and intervene in the processes that shape more ultimate forms of performance 
(Jobs for the Future, 2008). Washington State has pioneered the use of such intermediate 
outcomes in its Student Achievement Initiative for community colleges (Jenkins, Ellwein, 
& Boswell, 2009). 15 Much the same indicators are being tested by six states participating 
in the Cross-State Data Work Group of the Achieving the Dream initiative (Jobs for the 
Future, 2008). 

Beyond graduation, transfer, and job placement, it is also important to measure how well 
students are completing. Performance data should include indicators for post-transfer 
performance, passing licensure exams, employer satisfaction with graduates, and long 
term job stability. 

While indicators for specific kinds of learning are important, consideration should also be 
given to general student learning. Indicators of general learning are important to provide 
colleges with an incentive to foster students’ general education and avoid dealing with 
completion demands by watering down curricular requirements or engaging in grade 
inflation (Dougherty & Hong, 2006). Moreover, the inclusion of direct measures of 
learning may be an important means to impart greater legitimacy to the Voluntary 
Framework of Accountability for Community Colleges, as it has had for the Voluntary 
System of Accountability for four-year colleges. Needless to say, devising reasonable 
measures of student general learning is very difficult, but it is clear that this issue will not 
go away, after having been forcefully put on the agenda by the Spellings Commission 
and numerous other commentators (Erisman & Gao, 2006; Ewell & Jones, 2006; 

National Commission for Accountability in Higher Education, 2005; U.S. Department of 
Education, 2006). Determining how best to measure general student learning is a 
contentious issue and may account for the absence of general learning indicators in state 
performance accountability systems. To best address this matter, those developing 
student learning indicators and their measures should make every effort to involve faculty 
in their design. Faculty are both key carriers of curricular and pedagogical expertise and 
key implementers of any learning program. Failure to involve them increases the 
probability of indicators and measures that lack legitimacy and technical rigor. 

Finally, just as the input indicators need to be broken down by student background, so do 
the output or outcome indicators. As noted in our findings, this is an area where the 
performance indicators in our 10 states are particularly deficient. Only 4 states 
disaggregate outputs by race/ethnicity and even fewer do so by gender, income, age, or 
language status. Again, including indicators that take into account less advantaged 
students is important in order to counteract any incentive community colleges might have 
to improve their outputs by focusing on more advantaged students (Dougherty & Hong, 
2006). Moreover, including such indicators avoids penalizing colleges that take in many 
disadvantaged students and have to invest in more extensive student support programs. 16 
All this said, it should not be extremely difficult for states to disaggregate at least some 



26 




outcomes by student background. A national survey by the National Center for Higher 
Education Management Systems in 2006 found that 62 percent of states had state student 
unit record data systems that contained longitudinal data for individual students giving 
their race/ethnicity, gender, credit completion, and graduation (Ewell & Boeke, 2006). 17 

Systemic indicators. Beyond these issues of selecting the right indicators of the 
performance of individual colleges, attention should also be given to securing good 
indicators of systemic performance. An important lesson that has emerged in recent years 
is the importance of indicators that capture the performance of the entire higher education 
system (Erisman & Gao, 2006; Ewell & Jones, 2006; National Commission on 
Accountability in Higher Education, 2005; Shulock, 2003; Wellman, 2006). Performance 
accountability systems need to measure not just how individual colleges are performing 
but also how well the entire higher educational system — community colleges and 
universities combined — is creating cultivated citizens and technically prepared workers. 
Such measurement is particularly important in the case of transfer to universities, where 
rates of transfer are shaped not just by what community colleges do to prepare students 
but also what the universities do to admit students. 

Ways to better measure indicators 

In order to yield data that validly measure how community colleges are performing, it is 
important that the indicators above be properly measured and that the resulting data be 
put in the right context. 

Proper measurement. In terms of proper measurement, it is important that the data 
pertain to the relevant students and fully cover their performance. Data should be 
gathered on all students and not just on those who are attending full time or began their 
enrollment in the fall. Moreover, for data on degree completion, it is important that the 
data pertain to students who are first-time students in community college and are indeed 
degree seeking. Finally, every effort should be to count all students who transfer to a 
four-year college: those transferring with or without a degree, to private as well as public 
colleges, and to out-of-state as well as in-state institutions (Offenstein & Shulock, 

2009). 18 

Another key consideration in producing data that fairly represent community colleges is 
extending the time frame for tracking outcomes for students. Many students attend 
community colleges part time or have to begin by taking noncredit remedial courses and 
therefore do not complete a degree or demonstrate other successful outcomes within the 
three years mandated by the federal Graduation Rate Survey. 19 When students are instead 
followed up six years after entry, completion rates go up sharply (Jobs for the Future, 
2008; Offenstein & Shulock, 2009). 

Contextualization of data. Community colleges vary greatly in their student composition 
and the nature of their catchment areas. Hence, seeming differences in performance may 
be due in good part to differences in the characteristics of entering students. In order to 
better understand how community colleges are really performing, performance indicators 
for community colleges need to be contextualized or benchmarked, either by comparing 
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colleges to relevant peer groups or by statistically adjusting performance data for the 
entry characteristics of students (Carey & Aldeman, 2008; Dougherty & Hong, 2006; 
Dowd & Tong, 2007; Erisman & Gao, 2006; Zumeta, 2001). 20 As noted above, 3 of our 
10 states provide peer group comparisons but it would be very helpful to have such 
comparisons be the norm. California and Oklahoma provide useful examples of how 
performance data can be benchmarked (California Community Colleges, 2009; 

Oklahoma State Regents for Higher Education, 2009). In addition, some 
contextualization can be achieved by calculating separate completion rates for remedial 
versus non-remedial students and comparing colleges within each category. Doing so will 
tend to remove the penalty that colleges with high remedial populations suffer in their 
reported completion rates (Offenstein & Shulock, 2000). 

In addition, performance measures must take into account local labor market differences. 
They must acknowledge the difficulties of some colleges — particularly those in rural 
areas — in meeting demands for job placement in well-paying jobs (Dougherty & Hong, 
2006). 

Finally, data contextualization is needed to cushion small colleges against the distortive 
effects of small sample sizes, where changes in only a few students can have a dramatic 
effect on average performance. One way to counteract the effects of sampling fluctuation 
would be to utilize rolling averages (as is done in Tennessee and North Carolina). 

Ways to make collection of performance data easier for community colleges 

The responses from the community colleges we surveyed concerning the problems they 
encounter in collecting and reporting performance data to the state suggest that certain 
initiatives would be helpful. 

First, data requirements should be tailored as much as possible to data already collected 
for state student unit record systems, which were present in 40 states as of 2006 (Ewell & 
Boeke, 2006). 

Second, community colleges would benefit from having the state itself take over more 
data collection tasks. Graduate and employer surveys might be better done by the state, 
given their high cost and the state’s greater resources for getting responses. Similarly, 
the colleges would be helped if the state collected information from “third parties,” such 
as the state university system (transfer data) and professional associations (licensing pass 
rates). 22 

Third, for data that they must still collect and analyze for themselves, community 
colleges would benefit from state-provided technical assistance and training, particularly 
in collecting data requiring surveys of graduates, employers, and others. 23 Moreover, 
many community colleges — particularly the smaller ones — could use assistance in 
acquiring better information technology capacity (Dougherty & Hong, 2006). As a 
Maryland local community college official noted, “some smaller colleges are hindered by 
limited IT infrastructure which may make data extraction and retrieval more difficult than 
for their large-college colleagues.” 
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Ways to encourage colleges and state officials to better use performance data 

Community colleges. Our interviews point to several ways that community colleges can 
be helped to do a better job of analyzing data on their performance. First, they need 
assistance in improving their information technology and institutional research capacity 
(in the form of better equipment and more and better trained staff) to do high quality data 
analysis (Morest & Jenkins, 2007). 

Second, community college constituents — particularly faculty and middle managers, 
who often ignore or are unaware of performance data — need to have incentives for 
paying greater attention to performance data. States could encourage the use of the 
Evidence Based Inquiry Councils that Alicia Dowd and Estela Bensimon have been 
calling for. Organized on the basis of consortia of community colleges, these councils 
could play an important role in fostering the skills and attitudes that aid community 
colleges in analyzing their performance, considering new programs to improve their 
performance, and testing how well those new programs work (Dowd & Tong, 2007). In 
addition, Joseph Burke and his associates have suggested that colleges be encouraged to 
require performance reporting by departments and other organizational sub-units. 
Moreover, colleges could be rewarded financially for taking performance data into 
account by having their use in internal budgeting be an indicator in state performance 
funding systems (Burke & Associates, 2002; Burke & Minassians, 2002). 24 

Third, local community colleges would benefit from more detailed commentary from 
state officials on the data that the colleges submit to the state. As a local official in 
Maryland noted, 

[W]e would like to see... more substantial review, feedback, analysis, and 
observation from the state and within the college on our reports and our 
performance. . ..It would be informative if the state would provide an 
indication of its expectations for the community colleges as a whole on 
some of the indicators that are tied to the State Plan for Higher Education. 

All of these resources for community colleges are made more effective if performance 
indicators are kept stable over several years. This stability allows local and state 
community colleges to better learn how well particular performance indicators are 
capturing the key educational processes of interest and how to respond programmatically 
to those findings (Dougherty & Hong, 2006; Erisman & Gao, 2006). 

State officials and other external stakeholders. One way to encourage state officials to 
make better use of performance data is to improve the number and skills of staff members 
in state offices who can analyze the data. But it is also important to get legislators and 
executive branch officials to pay more attention to data. It has been proposed that state 
officials be mandated to acknowledge that they have read and responded to state 
accountability information. In fact, state officials could be asked to detail precisely how 
they have taken performance data into account in improving policy and performance and 
how they will use them in the future (see Burke & Minassians, 2002). 
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NOTES 



1 Since the Dougherty and Reid (2007) report, Texas added performance funding for the state universities 
(Southern Regional Education Board, 2007) and Washington has done the same for community colleges 
(Davis, Ellwein, & Boswell, 2009; Washington State Board for Community and Technical Colleges, 2007). 
Arguably, the number of states with performance funding should not include those states that have 
established systems but have not funded them. Omitting those states would remove Louisiana and Oregon 
from the list and reduce the number to around 13. 

2 Washington State had a short-lived performance funding system between 1997 and 1999 (Dougherty & 
Natow, 2009). 

3 Today, about one quarter ( 15) of the states have performance funding, in which a portion of state 
appropriations are tied to institutional performance on specific indicators. Virtually all the states have 
performance reporting (Dougherty & Reid, 2007; Southern Regional Education Board, 2007). 

4 We differentiate indicators from measures. By measures we mean the particular way that an indicator is 
operationalized. For example, does the graduation indicator take the form of the gross number of graduates 
or of the rate of graduation? If the measure is the graduation rate, what is the denominator: all entrants to 
the community college or just those who have reached a certain milestone? 

5 Enrollments are not always present in state performance data systems because of the focus on outputs and 
outcomes. 

6 Arguably, we are actually overestimating the actual incidence of states breaking down performance data 
by student background. For example, we are taking enrollment in English as a Second Language programs 
as an example of breaking enrollments down by language status. Clearly, this is only a rough 
approximation to such a breakdown. See the Appendix for additional information. 

7 The CLAST was done away with in the last legislative session due to budget issues. 

8 The “High” response from North Carolina seems to be more a product of the fact that the state can easily 
report IPEDS data from its state unit-record system rather than an assessment of the degree to which the 
state performance data overlap the IPEDS data. 

9 However, it should be noted that, even when states publicly report performance data, these data may only 
comprise a subset of the total performance data collected. For example, Florida reports certain performance 
data only to the community colleges themselves and not to the public or to the legislature (Dougherty, Reid. 
& Nienhusser, 2006). 

10 The variables used for the cluster analysis vary by the indicator. For example, to determine the peer 
groups on the indicator for persistence, the cluster analysis used these three variables for each community 
college: student headcount enrollment; proportion of students age 25 and older; and median household 
income in a college’s Economic Service Area. Meanwhile, to determine the peer groups for the basic skills 
course completion indicator, the variables used for the cluster analysis were student headcount, 75 th 
percentile math SAT score at the nearest campus of the California University System, and the poverty rate 
in a college’s service area (California Community Colleges, 2009). 

1 1 A study of state community college policy in 5 states corroborated this, finding very limited state use of 
performance data to craft state policy in 4 of the 5 states (Dougherty et al., 2006). 

12 This variation shows up as well in the reports of local community college officials interviewed as part of 
a study of state community college policies in the first five Achieving the Dream states. Some colleges 
reported moderate to heavy use of state performance data; others reported little use. However, the modal 
pattern is of heavier use than at the state level (Dougherty et al., 2006). 

13 To be sure, 8 of the states we examined do break down enrollments by race and ethnicity and five by 
income, but only three do so by gender and one each by age and language status. 
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14 That said, breaking data down more often by student background is not always easy. There are reports of 
students increasingly refusing to report their ethnicity when asked. Also, income information is hard to 
collect because students feel it is intrusive to be asked about it. 

15 The Student Achievement Initiative of the Washington State Community and Technical College system 
has four intermediate achievement indicators. Two measure first year college-level progress in terms of the 
number of students earning the first 15 and first 30 college-level credits. Another indicator measures the 
number earning the first 5 college-level credits in a math class that meets the requirement for computation 
(applied degree) or quantitative reasoning (transfer degree). The fourth indicator measures gains in skill for 
students taking adult basic education or remedial education (Washington State Board for Community and 
Technical Colleges, 2007). 

16 In a study of the first year of the Student Achievement Initiative in Washington, community college 
respondents voiced concern that — because the Initiative does not have such indicators that take student 
background into consideration — colleges that take in more disadvantaged students are penalized (Jenkins 
et al., 2009). 

17 The figures are for states that collect data for the following variables defined as “set 2”: Sex, 
Race/Ethnicity, Date of Birth, Geographic Origin, Program/Major, Full-time/Part-time Status, Credits 
Attempted, Credits Earned, Degree Awarded (Ewell & Boeke, 2006, Table 3). 

18 In order to determine whether students are first-time to a community college, it has been recommended 
that gathering this information not be left to self report but rather checked in a national database, such as 
the National Student Clearinghouse. Similarly, determining degree intentions should also not be left to 
student self report but determined by whether students exhibit a combination of having taken more than six 
credits in the first year and enrolled within the first two years in a course (such as college math or English) 
that is usually taken only by degree seeking students. Finally, in order to capture all transfer students, it is 
also recommended that the National Student Clearinghouse be used, rather than relying just on reports from 
in-state public four-year colleges and universities (Offenstein & Shulock, 2009). 

19 IPEDS is now collecting data on outcomes four years, as well as three years, after community college 
entrance. 

20 The Job Training Partnership Act used such a system of statistical adjustment to set standards for service 
delivery areas (SDAs) (Bailey, 1988). However, a warning is necessary here. Though many students at 
community colleges do not aspire to a degree, colleges should still aim strongly to “warm up” aspirations, 
especially those of less-advantaged students. Adjusting retention and graduation measures for student body 
composition must not become a substitute for vigorous action to reduce rates at which students leave 
college without a degree or transferring to another institution. One way to encourage such vigorous action 
is to gradually increase the expected standard for retention and graduation, so that colleges are pushed to 
warm up student aspirations. 

21 This is no panacea, however. State agencies also run into significant problems in getting adequate 
responses rates to their surveys. 

~ 2 However, this means that state governors and legislators have to become more committed to providing 
state community college boards with more funds to improve their data gathering and analysis capacities. 

23 The importance of technical assistance has also been noted by Dougherty and Hong (2006), Grubb and 
Badway (2004), and Petrides (2002). 

~ 4 That said, it should be noted that states can expect significant resistance to indicators of intra-institutional 
use of performance data. Many community colleges may react to the suggestion in the text as badly 
exacerbating what they already perceive to be overly heavy reporting requirements. 
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APPENDIX: 

GENERAL AND SPECIFIC INDICATORS IN STATE PERFORMANCE 
REPORTING AND FUNDING SYSTEMS IN 10 STATES 



General Indicator 


Specific Indicator 


States 


Input Indicators 






Enrollments 


Fall headcount 


TX, MA, MD 


Enrollments 


Annual unduplicated enrollment 


TX, CA, MA, 
LA, NC, MD 


Enrollments 


Enrollment by semester 


TX 


Enrollments 


Recent high school graduates: Number 


NC 


Population served 


Population served (CA: system wide) 


IL, CA 


Population served 


Percentage of recent high school graduates enrolled in 
college 


FL, MD 


Population served 


Market share of first-time, full-time freshmen 


MD 


Population served 


Percentage of high school tech prep students enrolling in a 
community college 


NC 


Population served 


Percentage of adult population enrolled 


NC 


Population served 


Percentage of high school dropouts served by basic skills 
courses 


NC 


Enrollment by race 


Enrollments: Number by race 


CA, MD, IL, 
TX, MA, NC, 
FL 


Enrollment by race 


Population served: Rate by race (CA: system wide) 


CA, OR 


Enrollment by race 


Enrollment share of races compared to population share 


TX, MA, MD 


Enrollment by 
language status 


Enrollments: Number in English as Second Language 
courses 


MD 


Enrollment by 
income 


Students receiving financial aid: Number 


TX, IL, LA, 
MD, NC, FL 


Enrollment by 
income 


First generation college students: Number 


IL 


Enrollment by gender 


Enrollments: Number by gender 


IL, CA 


Enrollment by gender 


Population served: Rate by gender (CA: system wide) 


CA 


Enrollment by gender 


Enrollment share compared to population share 


TX 


Enrollment by age 


Enrollments: Number by age 


CA 


Enrollment by age 


Population served: Rate by age (CA: system wide) 


CA 


Enrollment by 
disability status 


Enrollments: Number by disability status 


IL 


Enrollment by 
academic preparation 


Mean ACT Composite Score 


LA 


Enrollment by 
developmental needs 


Enrollments: Number by developmental education needs 


MD 
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General Indicator 


Specific Indicator 


States 


Process Indicators 






Tuition and fees 


Tuition and fee revenue for 30 semester credit hours 


TX 


Tuition and fees 


Tuition and basic fees for a full-time in-district student 


IF 


Tuition and fees 


Undergraduate mandatory attendance fees (resident and non- 
resident) 


FA 


Tuition and fees 


Tuition and fees as a percent of tuition and fees at public 
four-year institutions 


MD 


Tuition and fees 


Tuition and fees as a percent of median family income 


MA 


Tuition and fees 


Tuition and fees compared to community colleges within 
geographic region 


OR 


State funding 


State dollars per FTE 


FA 


Expenditures 


Percentage of expenditures on instruction 


MD 


Expenditures 


Percentage of expenditures on instruction and selected 
academic support 


MD 


Expenditures 


Percentage of expenditures on institutional support 


TX 


Expenditures 


Expenditure per student 


MA, FA 


Financial other 


Fundraising from private sources 


MA 


Financial other 


Projects and initiatives that result in cost savings and more 
efficient use of system resources. 


MA 


Financial other 


Review of fiscal health: Measure of how long community 
college could operate using only reserve funds, if all other 
revenues stopped. 


MA 


Financial other 


Independent annual audit of institutional fiscal practices 


MA 


Faculty 

characteristics 


Full-time faculty: Number and percentage of all faculty 


TX 


Faculty 

characteristics 


Contact hours: Percentage taught by faculty 


TX 


Faculty 

characteristics 


Faculty: Breakdown by race 


MD, IF 


Faculty 

characteristics 


Administrative and professional staff: Breakdown by race 


MD, IF 


Faculty 

characteristics 


Baccalaureate/transfer faculty preparation 


IF 


Faculty/student ratio 


Faculty /student ratio (TX). Average class size (IF) 


TX, IF 


Transfer education 
programming 


Number of general education and major specific courses 
included in the Illinois Articulation Agreement 


IF 


Vocational 

programming 


Enrollment in workforce development courses 


MD, MA, NC 


Vocational 

programming 


Enrollment in continuing professional education leading to 
government or industry-required certification or licensure 


MD 


Vocational 

programming 


Number of courses/workshops conducted by business and 
industry center 


IF 
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General Indicator 


Specific Indicator 


States 


Vocational 

programming 


Enrollment in contract training courses and/or small business 
development 


MD, TX, NC, 

IL 


Vocational 

programming 


Number of business organizations provided training and 
services involving contract training and/or small business 
development 


MD, IL, NC 


Teacher preparation 


Teachers, administrators, and paraprofessionals trained: 
Number 


IL 


Teacher preparation 


Course sections given: Number 


IL 


Teacher preparation 


Collaborative partnerships with university teacher education 
programs or K-12 schools: Number 


IL 


Non-credit 

programming 


Enrollment in non-credit coursework 


IL, MD 


Remediation 

programming 


Enrollment in developmental, basic skills, and literacy 
courses: Number 


IL, MD, TX, 
NC 


GED programming 


Enrollment in General Equivalency Diploma program 


TX 


Online programming 


Enrollment in online courses 


LA, IL, MD, 
NC 


Online programming 


Number of distance learning courses 


LA, IL 


Dual enrollments 


High school students served through dual enrollment 
courses: Number 


OR, IL, TX, 
OH, EL 


Output/Outcome 

Indicators 






HS completion 


General Equivalency Diplomas (GEDs) and Adult High 
School diplomas awarded: Number 


EL, NC 


HS completion 


GED certificate applicants successful: Rate 


OR, IL 


HS completion 


GED completers enrolling in college level coursework: 
Number 


IL 


Remediation success 


Adult Basic Education Literacy Completion Points awarded: 
Number 


EL 


Remediation success 


Students who complete basic skill/ESL: Rate 


OR 


Remediation success 


Progress of basic skills students: Increase in level of basic 
skills: Number (CA, IL); Rate (CA, NC) 


CA, IL, NC 


Remediation success 


Students who completed some remedial student credit hours 
in the current year: Number 


OH 


Remediation success 


Students who completed developmental education: Number 


EL 


Remediation success 


Students who completed developmental education: Rate 


TX, EL, IL, 
MD, NC 


Remediation success 


Success rate of developmental students in subsequent college 
level course: Rate 


NC 


Remediation success 


Remedial credits earned versus attempted 


IL 
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General Indicator 


Specific Indicator 


States 


Credits earned 


Students who earned at least 15 semester credit hours at 
institution in current year: Number 


OH 


Credits earned 


Students who earned at least 30 semester credit hours: 
Number (MA, OH), Rate (CA) 


MA, CA, OH 


Credits earned 


Students who earned at least 45 semester credit hours in the 
current year: Number 


OH 


Credits earned 


College preparatory retention and success rates: Rate of AA 
degree seeking student who completed at least 1 8 college 
credits during the tracking period and completed all college 
preparatory requirements. 


FL 


Specific courses 
passed 


Students earning first 5 semester student credit hours of 
General Studies level math courses at institution in current 
year: Number 


OH 


Specific courses 
passed 


Students who successfully complete college level course in 
math, reading, and writing: Rate 


TX 


Specific courses 
passed 


Students who successfully complete credit vocational, basic 
skill, ESL courses: Rate 


CA 


Course completion 


Students enrolled in credit courses who earned the credit(s): 
Rate 


MA, TX 


Skill and knowledge 
acquisition 


Passed College Level Academic Skills Test after completing 
60 or more college credit hours: Rate 


FL [suspended 
this year] 


Skill and knowledge 
acquisition 


Marketable skills awards completers: Number 


TX 


Skill and knowledge 
acquisition 


Occupational Completion Points generated 


FL 


Retention 


Persistence/retention rate 


MD, MA 


Retention 


Retention: Fall to spring 


MA 


Retention 


Retention: Fall to fall: Rate 


TX, LA, CA, 
MA, NC, FL 


Retention 


Retention: Second Year: Rate 


TX, LA 


Graduation from CC 


Three, four, and six-year graduation: Rates 


TX, LA 


Graduation from CC 


Associate degrees and certificates awarded: Rate 


TX, CA, MA, 
LA, OR 


Graduation from CC 


Associate degrees and certificates awarded: Number 


TX, FL, NC, 
OH 


Graduation from CC 


Occupational program degrees and certificates: Number 


IL, MD, OR 


Graduation from CC 


Nursing and allied health: Number (TX, LA); Rate (OR) 


TX, OR, LA, 
FL 


Graduation from CC 


Associate of Arts in Teaching: Number 


TX 


Graduation from CC 


Completers of Educator Preparation Institutes: Number 


FL 


Graduation from CC 


Awards in Closing the Gaps critical fields: Number 


TX 
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General Indicator 


Specific Indicator 


States 


Graduation from CC 


Graduated or still enrolled in college: Rate 


TX, FL, CA, 
MD, NC 


Continuing education 


Continuing education after community college (not restricted 
to transfer to senior institution): Number 


FL, NC 


Transfer 


Transfer to senior institution: Number 


CA, FL, NC, 
OH, TX 


Transfer 


Transfer to senior institution: Rate 


TX, CA, OR, 
IL, MA, MD 


Transfer 


Students graduating from University of CA and CA State 
Universities who began at community colleges: Number 


CA 


Transfer readiness 


Students who took necessary courses to prepare to transfer: 
Rate 


CA 


Transfer readiness 


Students who attempt one year of semester credits followed 
by transfer to an University System of Ohio university or 
branch in the current year: Number 


OH 


After-transfer 

performance 


Student academic performance at transfer institutions: Rate 


MD, FL, NC 


After-transfer 

performance 


BA graduation of students who attended community college 
and transferred to Univ. of CA or CA State Univ.: Number 


CA (system 
wide) 


After-transfer 

performance 


Transfer to senior institution and return for a 2nd year 


OR 


Time to degree 


AA graduates who complete in 72 credit hours or less: 
Number 


FL 


Employment 


Graduates employed after graduation: Number 


FL 


Employment 


Graduates employed after graduation: Rate 


FL, TX, NC 


Employment 


Graduates employed or enrolled in further education: Rate 


IL, TX 


Employment 


Income after graduation 


CA (system 
wide) 


Employment 


Career program graduates employed in a related field: Rate 


MD, FL 


Employment 


At-risk youth employed after community college: Rate 


OR 


Employment 


Current/incumbent workers who retained employment after 
exiting community college: Rate 


OR, IL 


Employment 


Adult clients employed after adult workforce programs: Rate 


OR 


Employment 


Dislocated workers who obtained employment with at least 
80% of prior earnings: Rate 


OR 


Other economic 
outcomes 


Small Business Development Center pre-venture/start-up 
entrepreneurs with a completed business plan who start a 
business: Rate 


OR 


Other economic 
outcomes 


Businesses attracted or retained through college business and 
industry centers: Number 


IL 
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General Indicator 


Specific Indicator 


States 


Passing licensure 
exams 


Licensure/certification exam pass: Rate 


TX, MD, MA, 
OR, IL, NC, 
FL 


Student satisfaction 


Student leavers (graduates and/or non-graduates) reporting 
that primary reason for attending was met: Rate 


MD, NC 


Student satisfaction 


Student leavers’ (graduates and/or non-graduates) 
satisfaction with college quality (job preparation and/or 
preparation for transfer): Rate 


MD, IL, NC 


Employer satisfaction 


Employer satisfaction with career program graduates: Rate 


MD, IL, NC 


Employer satisfaction 


Employer satisfaction with contract training: Rate 


MD, NC, OR 


Customer satisfaction 


Customers rating their satisfaction with college’s customer 
service as “good” or “excellent”: Rate 


OR 


Accreditation 


Academic program accreditation: Rate 


LA 


College honors and 
activities 


Numbers of members of Phi Beta Kappa, awards to school, 
participants in service learning programs 


TX 


Success by race 


GEDs, Adult High School diplomas, Adult Basic Education 
Literacy Completion Points: Number by race 


FL 


Success by race 


Remediation passing: Rate by race 


FL 


Success by race 


Retention: Rate by race 


TX, MD 


Success by race 


Graduates of degree and certificate programs: Number by 
race 


TX, FL 


Success by race 


Graduates of degree and certificate programs: Rate by race 


MD, IL, TX 


Success by race 


Marketable skill award completers: Number by race 


TX 


Success by race 


Graduation or continuation of education: Rate by race 


TX 


Success by race 


Transfer: Rates by race 


MD, TX, FL 


Success by race 


Job placement: Rate by race 


TX 


Success by race 


Licensure passing for teachers: Rate by race 


TX 


Success by income 


GEDs, Adult High School diplomas. Adult Basic Education 
Literacy Completion Points: Number by income 


FL 


Success by income 


Graduates of degree and certificate programs: Number by 
income 


FL 


Success by gender 


Graduates of degree and certificate programs: Number by 
gender 


TX 


Success by gender 


Graduates of degree and certificate programs: Rate by gender 


IL 


Success by gender 


Marketable skill award completers: Number by gender 


TX 


Success by gender 


Graduation or continuation of education: Rate by gender 


TX 


Success by age 


At-risk youth: Enrolled at-risk youth who remained in or 
returned to school, or obtained their high school 
diploma/equivalent upon exiting the program: Rate 


OR 
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General Indicator 


Specific Indicator 


States 


Success by age 


Current/incumbent workers who retained employment after 
exiting community college: Rate 


OR, IL 


Success by age 


Adult clients employed after adult workforce programs: Rate 


OR 


Success by age 


Dislocated workers who obtained employment with at least 
80% of prior earnings: Rate 


OR 


Success by language 
status 


GEDs, Adult Fligh School diplomas, Adult Basic Education 
Literacy Completion Points: Number who tested into English 
for Academic Puiposes 


FL 


Success by language 
status 


Graduates of degree or certificate programs: Number who 
tested into English for Academic Puiposes 


FL 


Success by disability 
status 


GEDs, Adult Fligh School diplomas. Adult Basic Education 
Literacy Awards: Number disabled 


FL 


Success by disability 
status 


Graduates of degree or certificate program: Number disabled 


FL 



Sources 

California 

Florida 

Illinois 
Louisiana 
Maryland 
Massachusetts 
North Carolina 

Ohio 

Oregon 

Texas 



California Community Colleges (2009). 

Florida Community College System (2007, 2009); 
Dougherty, Reid, & Nienhusser (2006) 

Illinois Community College Board (2009) 

Louisiana Division of Administration (2009) 

Maryland Fligher Education Commission (2008) 

Massachusetts Department of Fligher Education (2008) 

North Carolina Community College System (2008); 
Dougherty, Reid, & Nienhusser (2006) 

Ohio Board of Regents (2009) 

Oregon Department of Community Colleges and Workforce 
Development (2008) 

Texas Fligher Education Coordinating Board (2009); 
Dougherty et al. (2006) 
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